Sensor for degraded visual environment

ABSTRACT

A sensing system. In some embodiments, the system includes a first imaging radio frequency receiver, a second imaging radio frequency receiver, a first optical beam combiner, a first imaging optical receiver, a second optical beam combiner, and an optical detector array. The first optical beam combiner may be configured to combine optical signals of the imaging radio frequency receivers. The second optical beam combiner may be configured to combine the optical signals of the imaging radio frequency receivers, and the optical signal of the first imaging optical receiver.

FIELD

One or more aspects of embodiments according to the present inventionrelate to sensing, and more particularly to a sensing system capable ofsensing in a degraded visual environment.

BACKGROUND

Imaging and ranging sensors in commercial and military operations have awide range of applications, and may be affected differently by differentcircumstances. For example, optical sensors that operate in the visibleor short-wavelength infrared parts of the spectrum may have relativelygood resolution, but their performance may be readily degraded by thepresence of dust or fog. The performance of radio frequency andlong-wavelength infrared sensors may be less easily degraded, but theirresolution may be inferior. Conditions in which such sensors are usedmay change rapidly; for example, a helicopter near the ground may raise,as a result of draft from its main rotor, a cloud of dust that mayobscure the view of the pilot and also of any optical electronic sensorsoperating in the visible part of the spectrum.

Thus, there is a need for a versatile sensor capable of operating in adegraded visual environment.

SUMMARY

According to an embodiment of the present invention, there is provided asensing system, including: a first imaging radio frequency receiver, asecond imaging radio frequency receiver, a first optical beam combiner afirst imaging optical receiver, a second optical beam combiner, anoptical detector array, a read out integrated circuit, and a processingcircuit, the first optical beam combiner being configured to combine: anoptical signal of the first imaging radio frequency receiver, and anoptical signal of the second imaging radio frequency receiver, thesecond optical beam combiner being configured to combine: the opticalsignal of the first imaging radio frequency receiver, the optical signalof the second imaging radio frequency receiver, and an optical signal ofthe first imaging optical receiver, the first imaging radio frequencyreceiver being configured to form, on the optical detector array, anoptical image of a radio frequency scene within a field of view of thefirst imaging radio frequency receiver, the second imaging radiofrequency receiver being configured to form, on the optical detectorarray, an optical image of an optical scene within a field of view ofthe second imaging radio frequency receiver, and the first imagingoptical receiver being configured to form, on the optical detectorarray, an optical image of an optical scene within a field of view ofthe first imaging optical receiver.

In some embodiments, the sensing system further includes a radiofrequency transmitter, configured to illuminate a radio frequency scenewithin the field of view of the second imaging radio frequency receiver.

In some embodiments, the sensing system further includes a rangingcircuit for measuring a first time of flight, between a radio frequencypulse emitted by the radio frequency transmitter and a signal, from theoptical detector array, corresponding to a reflection from the radiofrequency scene of the radio frequency pulse.

In some embodiments, the sensing system further includes: a secondimaging optical receiver, a third optical beam combiner, and an opticaltransmitter, configured to illuminate an optical scene within the fieldof view of the second imaging optical receiver, the third optical beamcombiner being configured to combine: the optical signal of the firstimaging radio frequency receiver, the optical signal of the secondimaging radio frequency receiver, the optical signal of the firstimaging optical receiver, and an optical signal of the second imagingoptical receiver, the second imaging optical receiver being configuredto form, on the optical detector array, an optical image of an opticalscene within a field of view of the second imaging optical receiver.

In some embodiments, the optical detector array is configured to operateat any time in one of: a first mode, in which the optical detector arraydetects optical signals in a first wavelength range, the firstwavelength range being entirely within the wavelength range from 1.2microns to 3 microns, and a second mode, in which the optical detectorarray detects optical signals in a second wavelength range, the secondwavelength range being entirely within the wavelength range from 0.2microns to 15 microns.

In some embodiments: the imaging radio frequency receivers have outputwavelengths entirely contained within the first wavelength range; thesecond optical beam combiner is configured: to transmit the opticalsignals of the imaging radio frequency receivers, and to reflect theoptical signal of the first imaging optical receiver; the first imagingoptical receiver is configured to transmit light within the secondwavelength range; and the second optical beam combiner has awavelength-dependent transmissivity, the transmissivity being: greaterthan 60% for a first wavelength, within the first wavelength range, andless than 40% for a second wavelength, within the second wavelengthrange.

In some embodiments: the third optical beam combiner is configured: totransmit: the optical signals of the imaging radio frequency receivers,and the optical signal of the first imaging optical receiver, and toreflect the optical signal of the second imaging optical receiver; andthe second imaging optical receiver is configured to transmit lightwithin the first wavelength range.

In some embodiments, the sensing system further includes a rangingcircuit for measuring a second time of flight, between an optical pulseemitted by the optical transmitter and a signal, from the opticaldetector array, corresponding to a reflection from the optical scene ofthe optical pulse.

In some embodiments, the sensing system further includes including animage separation circuit configured to generate, from a signal from theoptical detector array, a first image, corresponding to the reflection,from the radio frequency scene, of the radio frequency pulse, and asecond image, corresponding to the reflection, from the optical scene,of the optical pulse.

In some embodiments, the image separation circuit is further configured:to generate, from the signal from the optical detector array, a thirdimage, corresponding to optical emission from the optical scene togenerate, from the signal from the optical detector array, a fourthimage, corresponding to radio frequency emission from the radiofrequency scene.

In some embodiments: the optical detector array is configured to operateat any time in one of: a first mode, in which the optical detector arraydetects optical signals in a first wavelength range, the firstwavelength range being entirely within the wavelength range from 1.2microns to 3 microns, and a second mode, in which the optical detectorarray detects optical signals in a second wavelength range, the secondwavelength range being entirely within the wavelength range from 0.2microns to 15 microns; and the sensing system is configured to operate:with the optical detector array in the first mode during a first timeinterval, and with the optical detector array in the second mode duringa second time interval; and the image separation circuit is configuredto generate the third image from a first portion of the signal from theoptical detector array, the first portion corresponding to the secondtime interval.

In some embodiments, the image separation circuit is configured togenerate the first image from a second portion of the signal from theoptical detector array, the second portion corresponding to asub-interval of the first time interval in which the reflection, fromthe optical scene, of the optical pulse, is absent.

In some embodiments, the image separation circuit is configured togenerate the second image from a third portion of the signal from theoptical detector array, the third portion corresponding to asub-interval of the first time interval in which the reflection, fromthe radio frequency scene, of the radio frequency pulse, is absent.

In some embodiments, the sensing system further includes a displayconnected to the processing circuit.

In some embodiments, the processing circuit is configured to: receiveuser input to select from among the first image, the second image, thethird image and the fourth image, and to cause the display to displaythe selected image.

In some embodiments, the processing circuit is configured to display twoof: the first image, the second image, the third image, and the fourthimage, concurrently.

In some embodiments, the processing circuit is configured to display: aportion of one of: the first image, the second image, and the thirdimage, and the fourth image, a portion of another one of: the firstimage, the second image, and the third image, and the fourth image,concurrently.

In some embodiments, the processing circuit is configured to display: aportion of the first image, a portion of the second image, and a portionof the third image, and the fourth image, concurrently.

In some embodiments, the processing circuit is configured to display aportion of the first image, and, overlaid on the portion of the firstimage, text indicating a range corresponding to the first time offlight.

In some embodiments, the processing circuit is configured to display aportion of the third image, and, overlaid on the portion of the firstimage, text indicating a range corresponding to the second time offlight.

BRIEF DESCRIPTION OF THE DRAWINGS

Features, aspects, and embodiments are described in conjunction with theattached drawings, in which:

FIG. 1 is a block diagram of a sensing system, according to anembodiment of the present invention;

FIG. 2 is a block diagram of a radio frequency receiver, according to anembodiment of the present invention;

FIG. 3 is a block diagram of a radio frequency to optical converter,according to an embodiment of the present invention;

FIG. 4 is a block diagram of an optical local oscillator, according toan embodiment of the present invention;

FIG. 5 is a block diagram of a sensing system, according to anembodiment of the present invention; and

FIG. 6 is a block diagram of a sensing system, according to anembodiment of the present invention.

DETAILED DESCRIPTION

The detailed description set forth below in connection with the appendeddrawings is intended as a description of exemplary embodiments of asensor for a degraded visual environment provided in accordance with thepresent invention and is not intended to represent the only forms inwhich the present invention may be constructed or utilized. Thedescription sets forth the features of the present invention inconnection with the illustrated embodiments. It is to be understood,however, that the same or equivalent functions and structures may beaccomplished by different embodiments that are also intended to beencompassed within the scope of the invention. As denoted elsewhereherein, like element numbers are intended to indicate like elements orfeatures.

Referring to FIG. 1, in some embodiments, a sensing system includes anoptical telescope 105, a radio frequency antenna array 115, a radiofrequency to optical converter 120, an optical beam combiner 125,optical detector optics 130, an optical detector array 135, a read outintegrated circuit 127, a processing circuit 150 and a display 155. Inoperation, the optical telescope 105 may receive light from an opticalscene 156, this received light may be transmitted through the opticalbeam combiner 125 through the optical detector optics 130 (which mayinclude a lens) and focused on the optical detector array 135. Eachdetector in the optical detector array 135 converts the received lightinto electric charge. The read out integrated circuit 127 measures theelectric charge over a specified interval and outputs digital signalsproportional to the charge. The processing circuit 150 converts thisdigital signal into the proper format to create an optical image of thescene on the display 155. The optical detector array 135 mayperiodically be reset and the time interval between any such reset, anda subsequent read-out of the cumulative photon detections since thereset may be referred to as a “frame”.

In the embodiment of FIG. 1, the optical telescope 105 and the opticaldetector optics 130 may be considered to form an “optical receiver”. Asused herein, an “optical receiver” or a “radio frequency receiver” is asubsystem that receives and processes optical or radio frequencyradiation propagating in free space. An “imaging receiver” (e.g., animaging optical receiver or an imaging radio frequency receiver) is asubsystem that receives and processes the electromagnetic radiation in amanner making possible the reconstruction of an image of the scene. Assuch, the optical telescope 105 of FIG. 1 is an imaging opticalreceiver, and the combination of the optical telescope 105 and theoptical detector optics 130 of FIG. 1 can also form an imaging opticalreceiver. Similarly, the radio frequency antenna array 115, the radiofrequency to optical converter 120, and the optical detector optics 130together form an imaging radio frequency receiver. In the embodiment ofFIG. 1, the optical detector optics 130 form a shared element of (i) theimaging optical receiver that includes the optical telescope 105 and theoptical detector optics 130 and of (ii) the imaging radio frequencyreceiver that includes the radio frequency receiver 115, the radiofrequency to optical converter 120 and the optical detector optics 130.Similarly, in the embodiment of FIG. 1, the optical beam combiner 125 isa shared element of (i) the imaging optical receiver that includes theoptical telescope 105 and the optical beam combiner 125 and of (ii) theimaging radio frequency receiver that includes the radio frequencyantenna array 115, the radio frequency to optical converter 120 and theoptical beam combiner 125. As used herein, an “optical beam combiner” isany passive optical system that has at least two inputs and at least oneoutput, the light at the output being a linear combination of the lightat the inputs. It may be a partially silvered mirror, for example, or itmay include powered elements such as lenses. In some embodiments, it mayinclude, for example, a grating or prism to combine differentwavelengths.

The “optical scene”, as used herein, means the set of things (e.g.,buildings, vegetation, soil, rocks, vehicles, aircraft, sky, or clouds)from which light, in a wavelength range of interest, reaches thedetector through the optical receiver. Similarly, the “radio frequencyscene”, as used herein, means the set of things from whichelectromagnetic radiation, in a frequency range of interest, reaches thedetector through the radio frequency receiver, as discussed in furtherdetail below.

In the embodiment of FIG. 1, the radio frequency antenna array 115 mayinclude a plurality of antenna elements 205 as illustrated in FIG. 2.Referring to FIG. 3, in some embodiments, the radio frequency to opticalconverter 120 includes a corresponding plurality of low noise amplifiers210 and an array of optical modulators 305. Each optical modulator 305has an optical local oscillator input fed by an optical local oscillatorsignal (e.g., a signal from an optical local oscillator 415, discussedin further detail below), a modulation input, and an output. In someembodiments, each optical modulator 305 is a phase modulator, which mayinclude a nonlinear crystal (e.g., a lithium niobate crystal) the indexof refraction of which depends on an electric field applied across it.In operation, a radio frequency tone received by one of the antennaelements 205, amplified by one of the low noise amplifiers 210 and inputto one of the optical modulators 305 may cause phase modulation of theoptical local oscillator signal, resulting, at the output of the opticalmodulator 305, in a signal including a carrier component, an uppersideband, and a lower sideband. For large modulation depth, othersidebands may also be present, and the carrier may be suppressed (orentirely absent, if the modulation depth corresponds to a zero of thezeroth Bessel function of the first kind).

The phase of the upper sideband may be equal to the sum of the phase ofthe optical local oscillator signal and the phase of the radio frequencytone. The output of each phase modulator may be connected to a filter310 (e.g., a high-pass or band-pass filter) that allows the uppermodulation sideband to pass and rejects or blocks the carrier and thelower modulation sideband. As such, each of the modulators in such anembodiment acts as a phase-preserving frequency converter. An amplitudemodulator (e.g., an electro-absorption modulator or a Mach-Zehnderinterferometer having a phase modulator in each arm, the phasemodulators being driven in opposite directions by the radio frequencymodulating signal), similarly followed by a filter 310 that passes onemodulation sideband while blocking the carrier and the other modulationsideband, may similarly act as a phase-preserving frequency converter.Referring to FIG. 4, the optical local oscillator 415 may include, forexample, a laser 405 (e.g., a fiber-coupled laser) and a shuttermechanism 410 (which may be used for image separation, as discussed infurther detail below). In some embodiments, the optical local oscillator415 has a coherence length comparable to or greater than a maximumround-trip time of flight for radio frequency signals, and coherentactive radio frequency sensing may be performed by the sensing system.

Referring again to FIG. 1, the phase-preserving property of thephase-preserving frequency converters may make it possible to form, onthe optical detector array 135, an optical image of the radio frequencyscene 158. For example, near-planar radio frequency waves received bythe radio frequency antenna array 115 from a distant radio frequencypoint source may have a phase that varies nearly linearly across theantenna elements of the array antenna, with a phase slope across thearray antenna corresponding to the direction from which the wavesarrive. This phase slope may be preserved at the outputs of the radiofrequency to optical converter 120, causing the optical detector optics130 to focus the optical signal at the output of the radio frequency tooptical converter 120 to a single detector in the optical detector array135, the location of the point corresponding to the direction from whichthe radio frequency waves arrive at the radio frequency antenna array115.

The optical beam combiner 125 (and other optical beam combinersdescribed herein) may be an optical element (e.g., a flat transparentplate (composed of, e.g., fused silica, or, for long-wave infraredsignals, sodium chloride, potassium bromide, or another suitablematerial)) having a partially reflective coating on one surface, andconfigured to combine two optical beams propagating in free space. Suchan element may also be known as a “beam combiner”. In some embodimentsthe physical location of the optical channel and the radio frequencychannel relative to the optical beam combiner may be interchanged withrespect to the optical beam combiner 125, compared to the configurationof FIG. 1, so that the signal from the optical channel is transmittedthrough the optical beam combiner 125 and the signal from the radiofrequency channel is reflected from the optical beam combiner 125. Asused herein, the “optical channel” is the signal path from the input tooptical telescope 105 and through the optical beam combiner 125 and the“radio frequency channel” is the signal path from the radio frequencyantenna array 115 and through the optical beam combiner 125.

In some embodiments, the sensing system includes an optical transmitter140 (e.g., an infrared (IR) transmitter) to illuminate the optical scene156, or a radio frequency transmitter 145 (e.g., an X-band, 10 GHztransmitter radio frequency transmitter) to illuminate the radiofrequency scene 158, or both. Such a configuration may enable “active”sensing to be performed, instead of, or in addition to, the “passive”sensing that may be performed when the sensing system lacks atransmitter (e.g., an optical transmitter 140 or a radio frequencytransmitter 145). In such embodiments, the free space electromagneticradiation received by the optical telescope 105 may be a combination of(i) “natural” electromagnetic radiation from the optical scene 156(i.e., electromagnetic radiation that does not originate from within thesensing system, including, for example, electromagnetic radiationgenerated by objects in the optical scene (e.g., as thermal radiation,or by other radiation-generating processes) and electromagneticradiation generated elsewhere (e.g., sunlight, or cosmic microwavebackground radiation) and reflected from the optical scene 156 and (ii)electromagnetic radiation emitted by the optical transmitter 140 andreflected from the optical scene 156. Similarly, the free spaceelectromagnetic radiation received by the radio frequency antenna array115 may be a combination of (i) natural electromagnetic radiation fromthe radio frequency scene 158 and electromagnetic radiation generatedelsewhere and reflected from the radio frequency scene 158, and (ii)electromagnetic radiation emitted by the radio frequency transmitter 145and reflected from the radio frequency scene 158. In active sensingmodes the transmitted radiation may be used either for ranging (asdiscussed in further detail below) or to provide illumination (andthereby to increase the radiation received from the scene by the sensor)or both.

The optical detector array 135 array may be connected to a read outintegrated circuit 127 and a processing circuit 150 which may in turn beconnected to a display 155, or to a navigator 157, or to both. Thedisplay 155 may be for human-in-the-loop operation, and the navigator157 may be for autonomous operation. The processing circuit 150 mayreceive data from the detectors in the optical detector array 135 fromthe read out integrated circuit 127 and cause the display 155 to displayimages of the optical scene 156 or the radio frequency scene 158. Theterm “processing circuit” is used herein to mean any combination ofhardware, firmware, and software, employed to process data or digitalsignals. Processing circuit hardware may include, for example,application specific integrated circuits (ASICs), general purpose orspecial purpose central processing units (CPUs), digital signalprocessors (DSPs), graphics processing units (GPUs), and programmablelogic devices such as field programmable gate arrays (FPGAs). In aprocessing circuit, as used herein, each function is performed either byhardware configured, i.e., hard-wired, to perform that function, or bymore general purpose hardware, such as a CPU, configured to executeinstructions stored in a non-transitory storage medium. A processingcircuit may be fabricated on a single circuit wiring board (PCB) ordistributed over several interconnected PCBs. A processing circuit maycontain other processing circuits; for example a processing circuit mayinclude two processing circuits, an FPGA and a CPU, interconnected on aPCB.

In some embodiments, the optical detector array 135 may simultaneouslyreceive (i) signals corresponding to natural electromagnetic radiationfrom the optical scene 156 and electromagnetic radiation emitted by theoptical transmitter 140 and reflected from the optical scene 156 and(ii) signals corresponding to natural radio frequency emission from theradio frequency scene 158 and electromagnetic radiation emitted by theradio frequency transmitter 145 and reflected from the radio frequencyscene 158, and the displayed image may be a superposition of an image ofthe optical scene 156 and an image of the radio frequency scene, so thatany object that is bright either at optical wavelengths or at radiofrequencies will be bright in the displayed image.

In other embodiments, an image separation circuit may be employed toseparate images corresponding to different signals received by thesensing system. For example, in a passive sensing embodiment (in whichthe optical transmitter 140 and the radio frequency transmitter 145 areabsent or shut off), the image separation circuit may separate thesignal generated by the optical detector array 135 into signals from theradio frequency channel corresponding to natural electromagneticradiation from the radio frequency scene 158 and signals from theoptical channel corresponding to natural electromagnetic radiation fromthe optical scene 156, and display them (or portions of them) separatelyon the display 155. Such separation may be accomplished by any ofseveral methods.

For example, the wavelength bands of the optical signals received at theoptical detector array 135 (i) from the optical channel and (ii) fromthe radio frequency channel may differ. In this casewavelength-dependent detection may be used to perform image separationFor example, a controllable optical detector filter 170 (e.g., a filterwheel with two bandpass filters, each corresponding to one of the twowavelength bands) may be employed between the optical detector optics130 and the optical detector array 135 to allow only one of the twosignals to reach the detector at any time. Alternatively, the opticalpath from the optical telescope 105 may include an optical filter 165 toblock light in the wavelength band corresponding to the radio frequencychannel to radio frequency to optical converter 120 from reaching theoptical detector array 135. In operation, the processing circuit 150 mayacquire, from the optical detector array 135, a first image during afirst time interval, when a first filter (of the two filters) is in theoptical path, and a second image during a second time interval, when asecond filter (of the two filters) is in the optical path, and theprocessing circuit 150 (which may be, or which may include, the imageseparation circuit) may separate the images and cause the display 155 todisplay one or the other of the two images (e.g., as selected by a user)or to display the two images side by side or one above the other.

In a related embodiment, the optical detector array 135 may itself bewavelength selective, having for example an array of dual-band pixels,each of which is sensitive in a first wavelength range (e.g., shortwavelength infrared (SWIR), e.g., 1.2 microns-3.0 microns) and in asecond wavelength range (e.g., long wavelength infrared (LWIR), e.g., 8microns-15 microns), for a first bias condition and for a second biascondition, respectively. The detector may be operated in the first biascondition (with a first bias voltage or a first bias current applied toeach of the pixels) during a first time interval and in the second biascondition (with a second bias voltage or a second bias current appliedto each of the pixels) during a second time interval, and the processingcircuit 150 may then separate the images and, for example, display thecorresponding images separately on the display 155 (e.g., displaying oneor the other of the two images (e.g., as selected by a user) ordisplaying the two images side by side or one above the other).

In some embodiments, the wavelength band of the output of the radiofrequency to optical converter 120 may be selectable. This may beaccomplished by using a wavelength-selectable optical local oscillator415 in combination with an optical detector array 135 that is wavelengthselective. Such a wavelength-selectable optical local oscillator 415 mayinclude two lasers, configured to operate at different wavelengths, anda system for switching between them. For example, the outputs of the twolasers may be connected to respective inputs of an optical powercombiner, and the lasers may be configured so that only one operates ata time (and the other is shut off). In such an embodiment, the opticalfilter 165 may be absent. The controllable optical detector filter 170may be a filter wheel with two bandpass filters, each corresponding toone of the two wavelengths of operation of the optical local oscillator415. If the signal from the radio frequency channel is much strongerthat the signal from the optical channel (e.g. as a result of closeapproach to a very high power radio frequency source), the sensingsystem may switch the controllable optical detector filter 170 toanother wavelength band, switch the optical local oscillator 415 to fallwithin the newly selected wavelength band and switch the opticaldetector array 135 to the newly selected wavelength band. Operating in 2different wavelength bands prevents saturation of the detector by theradio frequency channel.

The optical beam combiner 125 may have a wavelength-dependenttransmissivity from the first input (the input corresponding to theradio frequency channel) to the output, the transmissivity being greaterthan 50% for the wavelength range of light from the radio frequencychannel and less than 50% for the wavelength range of light from theoptical channel. In some embodiments, the transmissivity from the firstinput to the output is greater than 75% for a first wavelength, withinthe wavelength range of the radio frequency channel, and thetransmissivity from the first input to the output is less than 25% for asecond wavelength, within the wavelength range of the optical channel.

In some embodiments, the optical local oscillator 415 may becontrollable (e.g., by the processing circuit 150) to be enabled ordisabled. For example, the optical local oscillator 415 may include ashutter mechanism 410 (FIG. 4) (e.g., an acousto-optic shutter, or amechanical shutter) that is controllable to shut off the optical localoscillator 415, or the pumping of the laser may be shut off (e.g., in asemiconductor laser, the drive current may be shut off) to shut off theoptical local oscillator 415. The optical path from the opticaltelescope 105 to the optical beam combiner 125 may also include anoptical controllable element 175 for preventing light from propagatingto the optical detector array 135; the optical controllable element 175may be a mechanical shutter, for example, or a rotatable mirror. Theoptical controllable element 175 may be controllable, in a first state,to open the shutter, or steer received light to the optical detectorarray 135, and, in a second state, to close the shutter, or steerreceived light away from the optical detector array 135 (and, e.g., ontoa beam dump). In such an embodiment, the sensing system may be operated,during a first time interval, with the optical controllable element 175in the first state and the optical local oscillator 415 disabled, and,during a second time interval, with the optical controllable element 175in the second state and the optical local oscillator 415 enabled. Theprocessing circuit 150 may then separate the corresponding images anddisplay the images separately on the display 155 (e.g., displaying oneor the other of the two images (e.g., as selected by a user) ordisplaying the two images side by side or one above the other).

In some embodiments the optical telescope (105 in FIG. 1) may be, forexample, a telescope with reflecting primary mirror 510 and reflectingsecondary mirror 515, as shown in FIG. 5. In some embodiments, the phasedelay through the channels of the radio frequency antenna array 115 andthe radio frequency to optical converter 120 may not all be the same,and, for example, the effect of the variation in phase delay may be toconvert received plane waves to converging spherical waves at the outputof the radio frequency to optical converter 120, so that the opticaldetector optics 130 may be unnecessary, or so that (i) the opticaldetector optics 130 may be selected for compatibility with the opticaltelescope 105, and (ii) light corresponding to a distant radio frequencysource may also be focused at the optical detector array 135.

In an embodiment with active sensing (i.e., with an optical transmitter140 or a radio frequency transmitter 145, or both), time divisionmultiplexing may be employed to separate the active images from eachother and from the passive images. For example, the minimum round-triptime of flight for a radio frequency signal may be the round-trip timeof flight for the nearest object in the radio frequency scene 158, andthe maximum round-trip time of flight for a radio frequency signal maybe the round-trip time of flight for the most distant object in theradio frequency scene 158. As such, if the “radio frequency reflectiontime interval” is the time interval that (i) begins later than theemission of a radio frequency pulse by the minimum round-trip time offlight and (ii) ends later than the emission of the pulse by the maximumround-trip time of flight, then the reflection from the radio frequencyscene 158 of the radio frequency pulse may be received by the sensingsystem entirely within the radio frequency reflection time interval, andthe reflection of the radio frequency pulse may be absent during a timeinterval preceding the radio frequency reflection time interval andduring a time interval following the radio frequency reflection timeinterval. Similarly, the reflection of an optical pulse may be absentduring a time interval preceding an analogous optical reflection timeinterval and during a time interval following the optical reflectiontime interval. Separation of the active images from each other may thenbe performed by transmitting each radio frequency pulse and each opticalpulse at respective times such that the radio frequency reflection timeinterval and the optical reflection time interval do not overlap; activeimages acquired by the optical detector array 135 during the radiofrequency reflection time interval may then be free of influence fromtransmitted optical pulses, and may be displayed or used to performradio frequency ranging (by measuring the time of flight). The activeimages may include, in addition to signals from the reflection of theradio frequency pulse, signals corresponding to natural radio frequencyand optical (passive) emissions. These contributions may be sufficientlysmall to be neglected, or their effects may be reduced by blocking ordisabling any input path not needed for the active image being acquired(e.g., putting the optical control element 175 in the second state whenan active radio frequency image is acquired), or corrections may be madefor their effects, for example, by subtracting from each active imageacquired by the optical detector array 135 during the radio frequencyreflection time interval a passive image obtained in otherwise similarcircumstances (e.g., with the optical control element 175 in the secondstate) during a time interval when reflections from transmitted pulsesare absent (i.e., outside of the radio frequency reflection timeinterval).

Active images acquired by the optical detector array 135 during theoptical frequency reflection time may similarly be displayed or used toperform optical ranging. Ranging (either radio frequency ranging oroptical ranging) may be performed with a ranging circuit. The rangingcircuit may include a timing circuit 160 for controlling the timing ofpulses transmitted by the optical transmitter 140 and the radiofrequency transmitter 145, and for controlling the optical detectorarray 135 and read out integrated circuit 127 to perform accurate timeof flight measurements. For example, the timing circuit 160 may send afirst control signal to the optical transmitter 140, to cause it totransmit an optical pulse. After the minimum round-trip time of flightfor an optical signal has elapsed, the timing circuit 160 may send asecond control signal to the processing circuit 150 and the read outintegrated circuit 127, to begin recording arrival times of photons, andafter the maximum round-trip time of flight for an optical signal haselapsed, the timing circuit 160 may send a third control signal to theprocessing circuit 150 and the read out integrated circuit 127, to stoprecording arrival times of photons. The time interval between the firstcontrol signal and the photon arrival times may be divided by 2 timesthe speed of light by the ranging circuit, to calculate the range to theoptical scene, for each detector in the optical detector array 135.

If the total number of received photons is relatively small, then therange estimation error for any pixel may be unacceptably large. In thiscase the arrival times from all of the detectors may be aggregated, andused to calculate a single (average) range for the entire optical scene.The ranging circuit may include the timing circuit 160, arrival-timeestimating circuits within the processing circuit 150 and some or all ofthe circuitry in the read out integrated circuit 127, which may be usedto perform corrections for passive background, perform rangecalculations, format the resulting range data for display, and the like.In such an embodiment the “image” generated by the ranging circuit maybe a single estimated range. In other embodiments the active imagegenerated by the sensing system may include range as a function ofposition on the optical detector array 135, or reflectivity as afunction of position on the optical detector array 135, or both.

The output of the read out integrated circuit 127 is referred to hereinas an “image” or a sequence of “images” regardless of the representationof this output. The output may be, for example, a two dimensional arrayrepresenting the electromagnetic radiation detected by the array ofdetectors of the optical detector array 135, or it may be a twodimensional array, each element of which is the time of arrival of aphoton detected, during an exposure, in the array of detectors in theoptical detector array 135 or it may be an average of the arrival timesdetected by all of the detectors in the optical detector array during aframe. As such, in the context of the image separation circuit, an imagemay be a single range value.

Different images (e.g., active or passive optical images and active orpassive radio frequency images) may be displayed on the display 155 invarious ways. In some embodiments, as mentioned above, user input may beused to select one such image at a time for display. In otherembodiments, the display 155 may display several images, or portions ofimages, simultaneously. For example, the display 155 may display thecurrent (i.e., most recently obtained) passive optical image in itsupper left quadrant, the current passive radio frequency image in itsupper right quadrant, the current active optical image in its lower leftquadrant, and the current active radio frequency image in its lowerright quadrant. In other embodiments, if certain portions of the radiofrequency scene or of the optical scene are of particular interest, thedisplay 155 may display the interesting portions on different parts ofthe display screen. For example, if a portion of the optical scene isobscured by cloud or fog, then a portion of the radio frequency scenecorresponding to that portion of the optical scene may be displayed, ina portion of the display 155 that otherwise might show the obscuredportion of the optical scene, were it not obscured. In some embodiments,images may be overlaid; for example, a passive optical image and apassive radio frequency image may be overlaid by summing the respectivepixel values, or text indicating a range value (obtained from an activeoptical image or an active radio frequency image) may be overlaid onanother image (e.g. on a passive optical image or on a passive radiofrequency image).

FIG. 6 is a block diagram of a sensing system for use in a degradedvisual environment, in some embodiments. The elements shown in FIG. 6have functions that are analogous to elements with the same referencesymbols in FIG. 1. The embodiment of FIG. 6 includes the first radiofrequency antenna array 115 for passive sensing (the lower radiofrequency array 115 of FIG. 6.) and the second radio frequency antennaarray 115 for active sensing (the upper radio frequency antenna array115 of FIG. 6) from the first and second radio frequency scenes,respectively, first and second radio frequency to optical converters120, connected to the first and second radio frequency antenna arrays115, and the first optical telescope 105 configured to collect passivelong-wave infrared or mid-wave infrared or short-wave infrared orvisible or ultraviolet radiation for passive optical sensing (the loweroptical telescope 105 of FIG. 6) and the second optical telescopeconfigured to collect short-wave infrared radiation for active opticalsensing (the upper optical telescope 105 of FIG. 6) from first andsecond optical scenes, respectively, first and second optical filters165 (e.g., band-pass filters for the long-wave infrared, mid-waveinfrared, short wave infrared or visible or ultraviolet radiation andshort-wave infrared radiation), connected to the first and secondoptical telescopes 105, respectively, first, second, and third opticalcontrollable elements 175 (connected to the second radio frequency tooptical converter 120, the first optical filter 165 and the secondoptical filter 165, respectively, first, second, and third optical beamcombiners 125 (connected to the first, second, and third opticalcontrollable elements 175, respectively), optical detector optics 130and a controllable optical detector filter 170 (shown combined in asingle block in FIG. 6), an optical detector array 135, a read outintegrated circuit 127, a processing circuit 150, a navigator 157, adisplay 155, a timing circuit 160, an optical transmitter 140 and aradio frequency transmitter 145. Such a sensing system may be employed,for example, in a degraded visual environment, which may exist, forexample, when a helicopter, at takeoff or landing, stirs up a cloud ofdust that obscures the view of the pilot (and of any electronic opticalsensors for sensing visible light).

In the embodiment of FIG. 6, the second radio frequency receiver 115,and the second radio frequency to optical converter 120 together form,on the optical detector array 135, an optical image of a radio frequencyscene within a field of view of the imaging radio frequency receiverthat includes the second radio frequency receiver 115 and the secondradio frequency to optical converter 120. The first optical telescope105 forms, on the optical detector array 135, an optical image of anoptical scene within a field of view of the first optical telescope 105.The second optical telescope 105 forms, on the optical detector array135, an optical image of an optical scene within a field of view of thesecond optical telescope 105. In some embodiments the three optical beamcombiners 125 are replaced with a single element, such as a suitablegrating, for combining the four beams into one beam that is then sent tothe optical detector array 135.

In the embodiment of FIG. 6, the optical detector array 135 may besensitive in two wavelength ranges, e.g., LWIR and SWIR. The activeradio frequency and active optical signals may be separated by timedivision multiplexing, as described above, for example. The LWIR channelmay receive radiation that propagates relatively well throughdust-filled air; as such, the LWIR image generated by the system of FIG.6 may remain available even in a degraded visual environment.

Although limited embodiments of a sensor for a degraded visualenvironment have been specifically described and illustrated herein,many modifications and variations will be apparent to those skilled inthe art. Accordingly, it is to be understood that a sensor for adegraded visual environment employed according to principles of thisinvention may be embodied other than as specifically described herein.The invention is also defined in the following claims, and equivalentsthereof.

What is claimed is:
 1. A sensing system, comprising: a first imagingradio frequency receiver, a second imaging radio frequency receiver, afirst optical beam combiner a first imaging optical receiver, a secondoptical beam combiner, an optical detector array, a read out integratedcircuit, and a processing circuit, the first optical beam combiner beingconfigured to combine: an optical signal of the first imaging radiofrequency receiver, and an optical signal of the second imaging radiofrequency receiver, the second optical beam combiner being configured tocombine: the optical signal of the first imaging radio frequencyreceiver, the optical signal of the second imaging radio frequencyreceiver, and an optical signal of the first imaging optical receiver,the first imaging radio frequency receiver being configured to form, onthe optical detector array, an optical image of a radio frequency scenewithin a field of view of the first imaging radio frequency receiver,the second imaging radio frequency receiver being configured to form, onthe optical detector array, an optical image of an optical scene withina field of view of the second imaging radio frequency receiver, and thefirst imaging optical receiver being configured to form, on the opticaldetector array, an optical image of an optical scene within a field ofview of the first imaging optical receiver.
 2. The sensing system ofclaim 1, further comprising a radio frequency transmitter, configured toilluminate a radio frequency scene within the field of view of thesecond imaging radio frequency receiver.
 3. The sensing system of claim2, further comprising a ranging circuit for measuring a first time offlight, between a radio frequency pulse emitted by the radio frequencytransmitter and a signal, from the optical detector array, correspondingto a reflection from the radio frequency scene of the radio frequencypulse.
 4. The sensing system of claim 3, further comprising: a secondimaging optical receiver, a third optical beam combiner, and an opticaltransmitter, configured to illuminate an optical scene within the fieldof view of the second imaging optical receiver, the third optical beamcombiner being configured to combine: the optical signal of the firstimaging radio frequency receiver, the optical signal of the secondimaging radio frequency receiver, the optical signal of the firstimaging optical receiver, and an optical signal of the second imagingoptical receiver, the second imaging optical receiver being configuredto form, on the optical detector array, an optical image of an opticalscene within a field of view of the second imaging optical receiver. 5.The sensing system of claim 4, wherein the optical detector array isconfigured to operate at any time in one of: a first mode, in which theoptical detector array detects optical signals in a first wavelengthrange, the first wavelength range being entirely within the wavelengthrange from 1.2 microns to 3 microns, and a second mode, in which theoptical detector array detects optical signals in a second wavelengthrange, the second wavelength range being entirely within the wavelengthrange from 0.2 microns to 15 microns.
 6. The sensing system of claim 5,wherein: the imaging radio frequency receivers have output wavelengthsentirely contained within the first wavelength range; the second opticalbeam combiner is configured: to transmit the optical signals of theimaging radio frequency receivers, and to reflect the optical signal ofthe first imaging optical receiver; the first imaging optical receiveris configured to transmit light within the second wavelength range; andthe second optical beam combiner has a wavelength-dependenttransmissivity, the transmissivity being: greater than 60% for a firstwavelength, within the first wavelength range, and less than 40% for asecond wavelength, within the second wavelength range.
 7. The sensingsystem of claim 6, wherein: the third optical beam combiner isconfigured: to transmit: the optical signals of the imaging radiofrequency receivers, and the optical signal of the first imaging opticalreceiver, and to reflect the optical signal of the second imagingoptical receiver; and the second imaging optical receiver is configuredto transmit light within the first wavelength range.
 8. The sensingsystem of claim 4, further comprising a ranging circuit for measuring asecond time of flight, between an optical pulse emitted by the opticaltransmitter and a signal, from the optical detector array, correspondingto a reflection from the optical scene of the optical pulse.
 9. Thesensing system of claim 8, further comprising an image separationcircuit configured to generate, from a signal from the optical detectorarray, a first image, corresponding to the reflection, from the radiofrequency scene, of the radio frequency pulse, and a second image,corresponding to the reflection, from the optical scene, of the opticalpulse.
 10. The sensing system of claim 9, wherein the image separationcircuit is further configured: to generate, from the signal from theoptical detector array, a third image, corresponding to optical emissionfrom the optical scene to generate, from the signal from the opticaldetector array, a fourth image, corresponding to radio frequencyemission from the radio frequency scene.
 11. The sensing system of claim10, wherein: the optical detector array is configured to operate at anytime in one of: a first mode, in which the optical detector arraydetects optical signals in a first wavelength range, the firstwavelength range being entirely within the wavelength range from 1.2microns to 3 microns, and a second mode, in which the optical detectorarray detects optical signals in a second wavelength range, the secondwavelength range being entirely within the wavelength range from 0.2microns to 15 microns; and the sensing system is configured to operate:with the optical detector array in the first mode during a first timeinterval, and with the optical detector array in the second mode duringa second time interval; and wherein the image separation circuit isconfigured to generate the third image from a first portion of thesignal from the optical detector array, the first portion correspondingto the second time interval.
 12. The sensing system of claim 11, whereinthe image separation circuit is configured to generate the first imagefrom a second portion of the signal from the optical detector array, thesecond portion corresponding to a sub-interval of the first timeinterval in which the reflection, from the optical scene, of the opticalpulse, is absent.
 13. The sensing system of claim 12, wherein the imageseparation circuit is configured to generate the second image from athird portion of the signal from the optical detector array, the thirdportion corresponding to a sub-interval of the first time interval inwhich the reflection, from the radio frequency scene, of the radiofrequency pulse, is absent.
 14. The sensing system of claim 13, furthercomprising a display connected to the processing circuit.
 15. Thesensing system of claim 14, wherein the processing circuit is configuredto: receive user input to select from among the first image, the secondimage, the third image and the fourth image, and to cause the display todisplay the selected image.
 16. The sensing system of claim 14, whereinthe processing circuit is configured to display two of: the first image,the second image, the third image, and the fourth image, concurrently.17. The sensing system of claim 14, wherein the processing circuit isconfigured to display: a portion of one of: the first image, the secondimage, and the third image, and the fourth image, a portion of anotherone of: the first image, the second image, and the third image, and thefourth image, concurrently.
 18. The sensing system of claim 17, whereinthe processing circuit is configured to display: a portion of the firstimage, a portion of the second image, and a portion of the third image,and the fourth image, concurrently.
 19. The sensing system of claim 14,wherein the processing circuit is configured to display a portion of thefirst image, and, overlaid on the portion of the first image, textindicating a range corresponding to the first time of flight.
 20. Thesensing system of claim 14, wherein the processing circuit is configuredto display a portion of the third image, and, overlaid on the portion ofthe first image, text indicating a range corresponding to the secondtime of flight.